In recent times, camera based applications have gained popularity in numerous fields such as security systems, traffic surveillance, robotics, autonomous vehicles, etc. The camera calibration is imperative in running machine vision-based applications. The camera calibration is a process of obtaining camera parameters to determine (mathematically and accurately) how a three-dimensional (3D) environment is projected onto the camera's two-dimensional (2D) image plane without being affected by any lens distortion. The camera parameters may be, for example, a focal length, a skew, a distortion, etc. Typically, the camera parameters are determined by capturing multiple images of a calibration pattern from different views. The projections of certain key points in the calibration pattern (such as, inner corners in case of a checkerboard pattern) are then detected on the captured images. Then the projected key points of the calibration pattern are used by a conventional camera calibration algorithm for calibrating the camera. There are various mathematical models, for example, an OpenCV pinhole camera model (OpenCV Dev Team, 2016, Camera Calibration and 3D Reconstruction; available at: http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) for cameras with a narrow field-of-view, a OCam-Calib model (Davide Scaramuzza, 2006, OCamCalib: Omnidirectional Camera Calibration Toolbox for Matlab; available at: https://sites.google.com/site/scarabotix/ocamcalib-toolbox) for catadioptric and fisheye cameras, etc., which use different kinds of camera parameters for camera calibration.
As mentioned above, the most widely used camera calibration methods process images taken from multiple views of a calibration pattern. However, capturing a sequence of such images may take too long and may be too complicated to fit into a mass production factory. Camera calibration algorithms typically require about 10-30 images of a calibration pattern. Acquiring multiple images and appropriately repositioning the calibration pattern (or the camera) multiple times after taking a picture is time-consuming, and requires undivided attention of a camera operator. Conventional pattern detection algorithms employ corner detection to locate a calibration object within the captured image. These pattern detection algorithms are designed to detect only a single board containing a particular calibration pattern. Additionally, the detection often fails due to illumination variation and noise present during the image capturing process.
One example of a calibration pattern typically used for calibrating cameras is a checkerboard. Corners and edges of the checkerboard are two most important features. Typical methods used for detecting corners of checkerboards include Harris & Stephens corner detection algorithm, smallest univalue segment assimilating nucleus (SUSAN) corner detection algorithm, X-corner detection algorithm, etc. Hough transformation may be used on the edges to identify a proper set of lines and to locate the checkerboard pattern. Another approach for locating a checkerboard is based on calculating a count of internal holes in an image of a checkerboard for a particular size of the checkerboard. Morphological operations may be applied on the input image for detecting contours and a hierarchical tree is built from the contours. The checkerboard is considered to be correctly identified when a contour having a predetermined number of holes is found. Another widely used calibration pattern is of ellipses, however corners and lines are not present in that case.
Autonomous vehicles operating with minimal human intervention may be used in transporting people and objects. Typically, some autonomous vehicles require an initial input from an operator, while some other designs of the autonomous vehicles are under constant operator control. Some autonomous vehicles can be operated entirely by remote. Conventional autonomous vehicles are equipped with multiple cameras for facilitating control of operation of the autonomous vehicle. Hence, each camera is to be calibrated to ensure reliable and secure operation of the autonomous vehicle.
A method is disclosed in U.S. Pat. No. 6,768,509 B1 for determining points of interest on an image of a camera calibration object, preferably a checkerboard-pattern. Contours of the calibration shapes are extracted to this end, however, the known method is not suitable for calibrating with multiple calibration patterns.
Calibration of a multi-camera system has already been disclosed in US 2016/0073101 A1. The disclosed multi-target camera calibration is achieved by using multiple cameras that capture one or more images of multiple targets. Images of multi-board targets may also be captured. However, an image of only one checkerboard is captured at a time. These known techniques include stitching together each checkerboard of the multi-board target together by identifying individual boards by their location relative to other boards. Each checkerboard of the multi-board target may be captured individually, and subsequently suppressed while other checkerboards are captured. Then, checkerboard images may be stitched together for calibration based on their relative positions. Thus, the known system does not eliminate the above mentioned deficiencies of individual pattern processing.
Once multiple calibration patterns are captured in an image, and the positions of these patterns in the image are determined, the camera can be calibrated by means of known techniques, e.g. by means those disclosed in:    Zhengyou Zhang (2009); A flexible new technique for camera calibration; IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), pp. 1330-1334;    Jean-Yves Bouguet (2015); Camera Calibration Toolbox for Matlab; available at: http://www.vision.caltech.edu/bouguetj/calib doc/;    Scaramuzza, D., Martinelli, A. and Siegwart, R. (2006); A Toolbox for Easy Calibrating Omnidirectional Cameras; Proceedings to IEEE International Conference on Intelligent Robots and Systems (IROS 2006).
However, the prior art is deficient in techniques that would improve speed of camera calibration by allowing a quick and reliable position-determining for multiple patterns in a captured image, especially for autonomous vehicles during assembly in mass manufacturing. The prior art is also deficient in techniques that improve robustness to noise and illumination variations during the camera calibration.